Resiliency in IO Tools for Scientific Computing

IO tools in numerical simulations often serve two functions, one for writing and reading files to restart calculations and the other for writing and processing diagnostic files including files for graphics post-processing. For diagnostic files, tools directly working for high-level data structures are desired, such meshes with adaptive mesh refinement, unstructured meshes, and association between meshes and variables. Typically, one file in numerical simulations involves several or many writings. One of questions about the resiliency of IO tools is the following. If a simulation was crashed during writing a file, could an IO tool read the high-level data structures, meshes and their associated variables in the file that was incompletely written? This paper describes how a parallel IO library, HIO, is developed and modified toward resiliency. The goal of the library is to provide sustainable, interoperable, efficient, scalable, and convenient tools for parallel IO and data management for high-level data structures in numerical simulations. The high-level data structures include multi-dimensional arrays, structured meshes, unstructured meshes, the meshes generated through adaptive mesh refinement, variables associated with these meshes, and data defined on particles in particle simulations. The library is based on MPI-IO. Compared with MPI-IO, the overhead to write the explicit users' data structures are very small. To further improve IO performance, in addition to meta data, the library could buffer problem-size data while keeping users' explicit highlevel data structures. The buffering mechanism improves IO performance by a factor 10 to 20 in simulations of multiphysics. For resiliency, we have designed and implemented additional procedure so that the library could read all the high-level data within a file even if the file was incompletely written due to an interruption of a simulation. The cost for the resiliency of the library is very small in either performance or space.